The group does not end up at a house or on the road or at a castle but in a garden, at work where new seeds can grow, yield produce and perhaps enhance the quality of life. As members of a small group of individuals away from the world's corruption, they can each have a personal task as well as set and reach goals together. This, after all, is what society is: A group of individuals with similar values and beliefs that are working for the common good. The object is to try and destroy the weeds that will do their best to choke and eat away at the seedlings, so the plants can grow and provide food, shelter, clothing and other necessities.

Despite the horrors that all of them have seen and individually faced, they know that boredom, doing nothing, is a worst fate of all. The woman asks rhetorically if it is worse to be raped scores of times by pirates, have a buttock cut off, run the from the Bulgarians, and be flogged and hanged in the galleys -- that is, to go through all these horrors they have experienced -- or stay where they were and do nothing? The new gardeners also know that growing crops is unlike the destruction and suffering...

It does not involve using people for their own economic gains or pleasure. In addition, they know that poverty is another thing to avoid at all costs. It puts people in compromising positions.
The garden does not necessarily have to be exactly just a garden, either, since Candide is an allegory. The garden can stand for other things that people do together. It can be creative activities such as music, dance and theater. or, it can be coming together for education in a school or other learning center. It can be joining as a societal unit for political reasons. or, it can mean helping one another, especially those in need, as volunteers. In a society, the soil can be cultivated in many ways. That is only way it can survive.